6.2 Web Services

  1. Motivations
    • Examples of service servers - Google Maps service; PayPal service; ...
    • Overall diagram
    • How to use your Google Account to sign in to other sites or apps? 1-1-m model? 1-m model?
  2. Learning objectives
    • List the three components in Web Services.
    • Explain how RESTful web services are different.
    • List the two types of APIs.
    • List the four REST operations with their corresponding HTTP methods.
    • List the naming conventions used in RESTful APIs.
    • Understand The Same-Origin Policy and Cross-Origin Resource Sharing (CORS)
    • Problem solving: implementation of a REST service that provides the use of MongoDB
      • Define the RESTful API between the client and the server.
      • Define the library API that is used for the client.
      • Implement a web app with the libray, and a server-side program.
      • Use Express middleware.
  3. Problem to solve - MongoDB web service
    • Let's develop a simple MongoDB web service.
    • What kind of services?
      • A user can access a MongDB database stored on the server from a web application. The user can insert a document in a collection. The user can find documents, update documents and delete documents.
    • Requirements?
      • Requests use RESTful API and responses use JSON strings.
      • Open a connection with username and password.
      • Connection should be maintained only for a certain time, e.g., 1 minute.
      • The connection needs to be refreshed at every transactions.
      • Operations: open, insert, find, update, delete, close
  4. Web services
    • Read severl paragraphs in Web Services.
      • How do you define web services?
        Web app components that provide certian information through open protocols
      • What language is used to describe web services?
      • What protocols are used access web services?
      • List the two types of uses of web services.
    • Here is another good reference - tutorialpoint - Web Services.
      • Web services are open standard based web applications that interact with other web applications for the purpose of exchanging data.
      • Questions
        • Which protocol(s) is(are) used?
        • Where can you find web service descriptions?
        • How to use web services?
        • Is automation possible?
  5. Representational State Transfer (REST)ful web services
  6. Example - Use your Google Account to sign in to other sites or apps
  7. Example - Google Maps API Web Services
    • An example of the 1-m (a client - multiple servers) model, where the web service is used by the client-side.
    • How to use Google Maps web services, not how to develop the web services
    • Read all in Google Maps JavaScript API - Overview.
      • This tutorial includes the API, how to use Google Maps web service, from the perspective of client-side programming. The API between the client and the server is not explained.
      • Here is an example.
      • Here is the source code. 
        <!DOCTYPE html>
<html>
  <head>
    <style type="text/css">
      html, body, #map-canvas { height: 100%; margin: 0; padding: 0;}
    </style>
    <script 
      src="https://maps.googleapis.com/maps/api/js?language=en?key=YOUR_API_KEY">  <!-- Assigned to my Google account -->
                                                                                          <!-- Look closely. Path? -->
    </script>
  </head>
  <body>
    <div id="map-canvas"></div>
    <script type="text/javascript">
      function initialize() {
        var mapOptions = {
          center: { lat: 50.671441, lng: -120.36273},  // TRU
          zoom: 14
        };
        var map = new google.maps.Map(document.getElementById('map-canvas'), mapOptions);
      }
      google.maps.event.addDomListener(window, 'load', initialize);
    </script>
  </body>
</html>
      • Let's try to understand the above code.
        • Use of a JavaScript library for the client-side API, in which the API between the client and the server is implemented
        • API key for Google services
        • Creation of a related object
      • Google Maps API requires API keys. You can obtain the API keys - Get the API key.
    • Here is another excellent tutorial - Google Maps Tutorial.
    • There are many very interesting features.
    • Does Google Maps service use any complex protocol or description?
    • Here is the full API reference - Google Maps Javascript API V3 Reference.
  8. How to develop a web service - what do we need to consider?
    • The services that will be provied by the web service server
    • The RESTful API between the client and the server
    • Implementations
      • Cross-Origin Resource Sharing (CORS) - A request for a resource (like an image or a font) outside of the origin is known as a cross-origin request. 'Outside of the origin' includes not only different server computer systems but also differet port numbers. Web browsers generally blocks the traffic coming from outside of the origin for security. The idea of CORS is to use HTTP header options to allow cross-origin resource requests. Read CORS Tutorial: A Guide to Cross-Origin Resource Sharing for further understanding.
      • Client-side
        • A libray that hides all the detail client-side implementation
        • The API how to use the above libray
      • Server-side
        • Support RESTful API
        • Support CORS
      • Trial 0. CORS - We will have an exercise later.
  9. RESTful APIs
  10. MongoDB web service
    • Let's develop a simple DB service.
    • What kind of services?
      • A user can access a MongDB database stored on the server from a web application. The user can insert a document in a collection. The user can find a document, update documents and delete documents.
      • To make the project simple, let's use one database for all users.
    • Requirements?
      • Requests use RESTful API and responses use JSON strings.
      • Open a connection with username and password.
      • Connection should be maintained only for a certain time, e.g., 1 minute.
      • The connection needs to be refreshed at every transactions.
      • Operations: open, insert, find, update, delete, close
    • Let's try this implementation. Note that you need to use http, not https, to run this example.
    • RESTful API
      • See 6.2.9 - Uniform interface
        • Request and response
        • Request - CRUD (Create, Read, Update, Delete) resource operations
          CreateHTTP POST
          ReadHTTP GET
          UpdateHTTP PUTNot supported in HTML5
          DeleteHTTP DELETENot supported in HTML5
        • Response - XML/JSON
        • Naming conventions for URIs that are used to locate resources
          • Nouns - E.g., /tokens/{id}, not /getToken
          • Pluralized resources - E.g., /tokens or /tokens/{id}/value for a singleton resouce
          • Forward slashes for hierachy - E.g., /tokens, /tokens/{id}, /tokens/{id}/value
          • Comma for lists - E.g., /tokens/{id1},{id2}
          • Query - E.g., /tokens?username=David
          • Lowercase letters and dashes - E.g., /tokens?username=David-Williams, not /Tokens?Username=David_Williams
          • No trailing forward slash - E.g., Not /tokens/
          • Meaningful names, no shortened names, not file extension, ...
      • Create/Register a new user
        • Request - POST /users?username=...&password=...
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Create/Open a new connection
        • Request - ??? /tokens?username=...&password=...
        • Response - '{"tokenid":"...", "explanaton":"..."}', where if tokenid is negative, then error
      • Create/Open a new collection or an existing collection
        • Request - ??? /collections?tokenid=...&name=...
        • Response - '{"collectionid":"...", "explanaton":"..."}', where if collectionid is negative, then error
      • Create/Insert a new document
        • Request - ??? /collections/{collection_id}?tokenid=...&document=..., where the document value should be a JSON string.
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Read/Find documents
        • Request - ??? /collections/{collection_id}?tokenid=...&query=..., where the query value should be a JSON string for MongoDB query.
        • Response - '{"result":"true"|"false", "documents":"...", "explanation":"..."}', where the document value is a JSON string of an array of found documents.
      • Update documents
        • Request - ??? /collections/{collection_id}?tokenid=...&query=...&document=...
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Delete documents
        • Request - ??? /collections/{collection_id}?tokenid=...&query=...
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Delete/Close the connection
        • Request - ??? /tokens/{tokenid}
        • Response - '{"result":"true"|"false", "explanation":"..."}'
      • Delete a user
        • Request - ??? /users?username=...&password=...
        • Response - '{"result":"true"|"false", "explanation":"..."}'
    • Cient-side library API
      • var db = new TRUMongoDBService(host) - Connect to the server
        • This object will hold
          • host
          • connection token id
          • collection name and id
        • How to use
          • If a user were not registered, then the user should be registered first.
          • .open()
          • .collection()
          • CRUD operations
          • .close()
      • db.register(username, password, callback(result)) - Register a new user
      • db.open(username, password, callback(result)) - Sign in
      • db.collection(collection_name, callback(result)) - Select a collection
      • Insert, find, update, delete: Operations can be strings of MongoDB operations.
        db.insert(document, callback(result)),
        db.find(query, callback(result)),
        db.update(query, document, callback(result))
        db.delete(query, callback(result)),
      • db.close() - Close the connection
      • Note that token_id and collection_id are not included in the above API
    • For the processing of RESTful API, let's study a Node framework, Express - Fast, unopinionated, minimalist web framework for Node.js, with a myriad of HTTP utility methods and middleware at your disposal, creating a robust API is quick and easy.
      • Basic routing: Routing refers to determining how an application responds to a client request to a particular endpoint, which is a URI (or path) and a specific HTTP request method (GET, POST, and so on). Each route can have one or more handler functions, which are executed when the route is matched. Route definition takes the following structure: 
        app.METHOD(PATH, HANDLER)
      • Hello World! 
        const express = require('express')
const app = express()
const port = 8087

// Allow Cross-domain requests, i.e., CORS (Cross-Origin Resource Sharing)
//   Why do we need this?
app.all('/*', function(req, res, next) {  // /*: any routes
    res.header("Access-Control-Allow-Origin", "*");
    res.header("Access-Control-Allow-Headers", "X-Requested-With");
    res.header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");  // Default: only GET and POST
    next();  // Express middleware function; To continue to next operations
});

// GET method route; what is a router here?
app.get('/', (req, res) => {
  res.send('GET: Hello World!')
})

// POST method route
app.post('/', (req, res) => {
  res.send('POST: Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
      • How to run the above example?
      • Trial 1. Let's run the above example using your port number, and test it. The example URLs to test can be http://cs.tru.ca:yourport/, http://cs.tru.ca:yourport/tests, and http://cs.tru.ca:yourport/tokens/1


      • How to check the query in the server? E.g., .../users?username=john&password=topsecret. Let's use body-parser in body-parser for POST queries. Here is an example. 
        const express = require('express')
const app = express()
const bodyParser = require('body-parser')  // for query with POST
const port = 8087

// For CORS
...

// GET method route; what is a router here?
app.get('/users', (req, res) => {
  res.send('GET: username=' + req.query.username + "; password=" + ????)
})

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }))

// POST method route
app.post('/users', (req, res) => {
  res.send('POST: username=' + req.body.username + "; password=" + ????)
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
      • Trial 2. Let's run the above example using your port number, and test it. The example URLs to test can be http://cs.tru.ca:.../users?username=john&password=topsecret


      • How to check parameters? E.g., .../tokens/1 and .../tokens/1,2. Let's req.params.id. Here is an example. 
        const express = require('express')
const app = express()
const port = 8087

// For CORS
...

// GET method route; what is a router here?
app.get('/tokens/:id', (req, res) => {
  res.send('GET: /tokens; id=" + req.params.id);
})

// POST method route
app.post('/tokens/:id', (req, res) => {
  res.send('POST: /tokens; id=" + req.params.id);
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
      • Trial 3. Let's run the above example using your port number, and test it. The example URLs to test can be http://cs.tru.ca:.../tokens/1


      • Read all in Basic routing.
    • Implementation of the server side code
      • Requirements
        • A new user can sign up.
        • An existing user needs to connect to the system, i.e., SignIn.
        • One collection for each user for the simplicity
        • In the beginning, the collection name is given from the user. The name of collection and username are not submitted for the next CRUD operations. Instead, a token id is submitted.
      • Collections in MongoDB
        • Users - triples of unique username and password and collection name
        • Tokens - count for token id, pairs of username and token id
        • How to find the collection name with a token id? Token id => username => collection name
      • Model to access MongoDB
        • model.js
        • API 
          ready(cb)
    true or false
close()
isValid(username, password, cb)
    true or false
register(username, password, cb)
    true or false
unsubscribe(username)
getNewToken(username, cb) 
    token id, or
    -1 (error case)
deleteToken(id, cb)
    true or false
collection(token_id, collection_name, cb)  // use collection_name for token_id
    (true, 1), or 
    (false, -1) or (false, -2)
insertOne(token_id, doc, cb) 
find(token_id, query, cb) 
    (true, result) or
    false
updateMany(token_id, query, newdoc, cb) 
    true or false
deleteMany(token_id, query, cb) 
    true or false
        • Example of SignUp: ready => register
        • Example of Unsubscribe: ready => unsubscribe
        • Example of SignIn/Connection: ready => isValid => getNewToken
        • Example of Close: ready => deleteToken => close
      • Let's use Express - Fast, unopinionated, minimalist web framework for Node.js. It says "With a myriad of HTTP utility methods and middleware at your disposal, creating a robust API is quick and easy."
      • Here is an example. 
        var express = require('express');
var bodyParser = require('body-parser');
var model = require('./model.js');

var app = express();

var server = app.listen(8087, function () {
    var host = server.address().address;
    var port = server.address().port;
    console.log("Listening at http://%s:%s", host, port)
})

// for POST query
app.use(bodyParser.urlencoded({ extended: true }));

// To support CORS
app.all('/*', function(req, res, next) {  // /*: any routes
    res.header("Access-Control-Allow-Origin", "*");
    res.header("Access-Control-Allow-Headers", "X-Requested-With");
    res.header("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");  // Default: only GET and POST
    next();  // Express middleware function; To continue to next operations
});

// Route operations

app.get('/', function (req, res) {  // HTTP GET method
    res.send("Welcome to TRU MongoDB Web Service!");
})

// SignUp
//   Request - POST /users?username=...&password=...
//   Response - '{"result":"true"|"false", "explanation":"..."}'
app.post('/users', function (req, res) {  // HTTP POST method
    var username = req.body.username;
    var password = req.body.password;
    console.log("username = %s, password = %s", username, password);
    model.ready(function(result) {
        if (result) {
            model.register(username, password, function(result) {
                if (result)
                    res.send(JSON.stringify({result:true, explanation:""}));
                else
                    res.send(JSON.stringify({result:false, explanation:"Registration error"}));
            });
        }
        else
            res.send(JSON.stringify({result:false, explanation:"Connection error"}));
    });
});

// ???
app.post(path/:id, function(req, res) {  // Several different post with different paths
    var id = req.params.id;
    .... 
});

// ???
app.get(path, function(req, res) { 
    var id = req.query.id;
    ....
});  

// ???
app.put(path, function(req, res) {  // HTTP PUT method
    var doc = req.body.document
    .... 
});

// ???
app.delete(path, function(req, res) { .... });  // HTTP DELETE method

...
      • Trial 4. Let's implement server.js that supports SignUp and Unsubscribe, and test it. Here are the related messages that submitted to the server. You can download model.js and use it.
        • Create/Register a new user - SignUp
          • Request - ??? /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Delete a user - Unsubscribe
          • Request - ??? /????username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Example of SignUp: ready => register
        • Example of Unsubscribe: ready => unsubscribe


      • Trial 5. Let's implement server.js that supports SignIn and Close, and test it. Here are the related messages that submitted to the server. You can download model.js and use it.
        • Create/Open a new connection - SignIn
          • Request - ??? /tokens?username=...&password=...
          • Response - '{"tokenid":"...", "explanaton":"..."}', where if tokenid is negative, then error
        • Delete/Close the connection - Close
          • Request - ??? /???/{tokenid}
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Example of SignIn/Connection: ready => isValid => getNewToken
        • Example of Close: ready => deleteToken => close


    • Implementation of the client side library
      • API
        • var db = new TRUMongoDBWebService(host) - Connect to the server
          • This object will hold
            • host
            • connection token id
            • collection name
            • collection id
          • How to use
            • If a user were not registered, then the user should be registered first.
            • .open()
            • .collection()
            • CRUD operations
            • .close()
        • db.register(host, username, password, function(result) {...}) - Register a new user
        • db.unsubscribe(host, username, password, function(result) {...}) - Delete a user
        • db.open(host, username, password, function(result) {...}) - Sign in
        • db.collection(collection_name, function(result) {...}) - Select a collection
        • Insert, find, update, delete: Operations can be strings of MongoDB operations.
          db.insert(document, function(result) {...}), // document and query are JSON strings of objects
          db.find(query, function(result) {...}), // result is an object
          db.update(query, document, function(result) {...})
          db.delete(query, function(result) {...}),
        • db.close() - Close the connection
        • Note that token_id and collection_id are not included in the above API
      • TRUMongoDBWebService() 
        function TRUMongoDBWebService() {
    this.host = "";  // with open()
    this.connection_token_id = -1;
    this.collection_name = "";
    this.collection_id = -1;
    
    this.register = function(host, username, password, callback) {
        ...
    }
    this.unsubscribe = function(host, username, password, callback) {
        ...
    }
    ...
}
      • register() 
            this.register = function(host, u, p, callback) {
        $.ajax({
            url: host + "/users",
            method: "post",
            data: { username: u, password: p },
            success: function(data) {
                data = JSON.parse(data);
                callback(data);
            }
        });
    }
      • Trial 6. Let's implement the above client-side library that supports SignUp and Unsubscribe, and test it. Here are the related messages that submitted to the server. (If you did not complete Trial 4 and 5, you may use tr6_server.js.)
        • Create/Register a new user - register()
          • Request - POST /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'
        • Delete a user - unsubscribe()
          • Request - DELETE /users?username=...&password=...
          • Response - '{"result":"true"|"false", "explanation":"..."}'


      • Trial 7. Let's implement the above client-side library that supports SignIn and Close, and test it. Here are the related messages that submitted to the server. (If you did not complete Trial 4 and 5, you may use tr6_server.js.)
        • Create/Open a new connection - open()
          • Request - POST /tokens?username=...&password=...
          • Response - '{"tokenid":"...", "explanaton":"..."}', where if tokenid is negative, then error
        • Delete/Close the connection - close()
          • Request - DELETE /tokens/{tokenid}
          • Response - '{"result":"true"|"false", "explanation":"..."}'


    • Implementation of the client side app
  11. Server-side parallelism: Read all in Improving Node.js Application Performance With Clustering

  12. References